Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BS: Implement revocation part of DB #2644

Merged
merged 6 commits into from
May 7, 2019

Conversation

lukedirtwalker
Copy link
Collaborator

@lukedirtwalker lukedirtwalker commented May 7, 2019

Adds the following methods to the BeaconDB:

  • AllRevocations returns all revocations in the DB
  • InsertRevocation adds or updates a revocation in the DB
  • DeleteRevocation delete a revocation
  • DeleteExpiredRevocations deletes all expired revocations

Changes the behavior of CandidateBeacons to only return beacons that have no live revocation.

Fixes #2550


This change is Reviewable

Copy link
Contributor

@oncilla oncilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 11 of 11 files at r1.
Reviewable status: all files reviewed, 6 unresolved discussions (waiting on @lukedirtwalker)

a discussion (no related file):
Add description



go/beacon_srv/internal/beacon/db.go, line 43 at r1 (raw file):

	BeaconSources(ctx context.Context) ([]addr.IA, error)
	// AllRevocations returns all revocations in the database as a channel. The
	// result channel either carries revocations or errors. After sending the

Should we close the channel on first error, or try to serve as many revocations as possible?


go/beacon_srv/internal/beacon/beacondbsqlite/db.go, line 148 at r1 (raw file):

				return
			}
			srev, err := path_mgmt.NewSignedRevInfoFromRaw(rawRev)

In general, this should not happen.
I wonder if we should really return here, or continue.

The receiver will have to drain the channel anyway, so it can just log and skip responses with errors in it. And return hopefully return more parsable revocations.


go/beacon_srv/internal/beacon/beacondbsqlite/db.go, line 482 at r1 (raw file):

	`
	return db.DoInTx(ctx, e.db, func(ctx context.Context, tx *sql.Tx) error {
		existingRev, err := containsNewerRev(ctx, tx, revInfo)

support for upsert would be nice.
Unfortunately we do not have a recent enough sqlite version


go/beacon_srv/internal/beacon/beacondbsqlite/db.go, line 503 at r1 (raw file):

	row := tx.QueryRowContext(ctx, query, revInfo.IA().I, revInfo.IA().A,
		revInfo.IfID, revInfo.RawTimestamp)
	err := row.Scan()

you can chain QueryRowContext and Scan


go/beacon_srv/internal/beacon/beacondbsqlite/db.go, line 524 at r1 (raw file):

	query := `
	DELETE FROM Revocations
	WHERE ExpirationTime < ?

<=


go/beacon_srv/internal/beacon/beacondbsqlite/schema.go, line 50 at r1 (raw file):

		IntfID INTEGER NOT NULL,
		LinkType INTEGER NOT NULL,
		RawTimestamp INTEGER NOT NULL,

This is a bit cryptic, I had to check the code to see what this is.
How about IssuingTime?

Copy link
Collaborator Author

@lukedirtwalker lukedirtwalker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 6 unresolved discussions (waiting on @oncilla)

a discussion (no related file):

Previously, Oncilla wrote…

Add description

Done.



go/beacon_srv/internal/beacon/db.go, line 43 at r1 (raw file):

Previously, Oncilla wrote…

Should we close the channel on first error, or try to serve as many revocations as possible?

Done.


go/beacon_srv/internal/beacon/beacondbsqlite/db.go, line 148 at r1 (raw file):

Previously, Oncilla wrote…

In general, this should not happen.
I wonder if we should really return here, or continue.

The receiver will have to drain the channel anyway, so it can just log and skip responses with errors in it. And return hopefully return more parsable revocations.

Done.


go/beacon_srv/internal/beacon/beacondbsqlite/db.go, line 482 at r1 (raw file):

Previously, Oncilla wrote…

support for upsert would be nice.
Unfortunately we do not have a recent enough sqlite version

+1


go/beacon_srv/internal/beacon/beacondbsqlite/db.go, line 503 at r1 (raw file):

Previously, Oncilla wrote…

you can chain QueryRowContext and Scan

Done.


go/beacon_srv/internal/beacon/beacondbsqlite/db.go, line 524 at r1 (raw file):

Previously, Oncilla wrote…

<=

No if I understand the Active function on RevInfo correctly the == still means active. But I fixed the comparison in the lookup.


go/beacon_srv/internal/beacon/beacondbsqlite/schema.go, line 50 at r1 (raw file):

Previously, Oncilla wrote…

This is a bit cryptic, I had to check the code to see what this is.
How about IssuingTime?

Done.

Copy link
Collaborator Author

@lukedirtwalker lukedirtwalker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 8 of 12 files reviewed, 7 unresolved discussions (waiting on @oncilla)


tools/gomocks, line 37 at r2 (raw file):

        (SCION_PACKAGE_PREFIX + "/go/beacon_srv/internal/beaconing",
            "BeaconInserter,BeaconProvider,SegmentProvider"),
        (SCION_PACKAGE_PREFIX + "/go/beacon_srv/internal/ifstate", "RevInserter"),

wrong pr sorry.

Copy link
Contributor

@oncilla oncilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 4 files at r2.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @lukedirtwalker)


go/beacon_srv/internal/beacon/db.go, line 53 at r2 (raw file):

	// result channel either carries revocations or errors. The error can
	// either be ErrReadingRows or ErrParse. After a ErrReadingRows occurs the
	// channel is closed, so the result might be incomplete. The channel must

s/, so/and


go/beacon_srv/internal/beacon/beacondbsqlite/db.go, line 137 at r2 (raw file):

	}
	res := make(chan beacon.RevocationOrErr)
	go func() {

I don't think this works well with sqlite.
rows will keep the sqlite internal lock open until they are closed.

For the beacons, I had to read everything in memory first to free the lock as soon as possible and get better performance.

Copy link
Collaborator Author

@lukedirtwalker lukedirtwalker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @oncilla)


go/beacon_srv/internal/beacon/db.go, line 53 at r2 (raw file):

Previously, Oncilla wrote…

s/, so/and

Done.


go/beacon_srv/internal/beacon/beacondbsqlite/db.go, line 137 at r2 (raw file):

Previously, Oncilla wrote…

I don't think this works well with sqlite.
rows will keep the sqlite internal lock open until they are closed.

For the beacons, I had to read everything in memory first to free the lock as soon as possible and get better performance.

Hm as discussed offline:
We don't really use this function except for testing and maybe an external tool so it shouldn't affect performance of the BS.

Copy link
Contributor

@oncilla oncilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 2 files at r3.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

Copy link
Contributor

@oncilla oncilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 3 of 3 files at r4.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

Copy link
Contributor

@oncilla oncilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 2 of 2 files at r5.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

@lukedirtwalker lukedirtwalker merged commit 749447f into scionproto:master May 7, 2019
@lukedirtwalker lukedirtwalker deleted the pubBSRevocationDB branch May 7, 2019 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BS: BeaconDB Insert/DeleteRevocations
2 participants